Optimal Stroke Learning with Policy Gradient Approach for Robotic Table Tennis

Learning to play table tennis is a challenging task for robots, as a wide variety of strokes required. Recent advances have shown that deep Reinforcement Learning (RL) is able to successfully learn the optimal actions in a simulated environment. However, the applicability of RL in real scenarios remains limited due to the high exploration effort. In this work, we propose a realistic simulation environment in which multiple models are built for the dynamics of the ball and the kinematics of the robot. Instead of training an end-to-end RL model, a novel policy gradient approach with TD3 backbone is proposed to learn the racket strokes based on the predicted state of the ball at the hitting time. In the experiments, we show that the proposed approach significantly outperforms the existing RL methods in simulation. Furthermore, to cross the domain from simulation to reality, we adopt an efficient retraining method and test it in three real scenarios. The resulting success rate is 98% and the distance error is around 24.9 cm. The total training time is about 1.5 hours

Adaptive Robot Systems in Highly Dynamic Environments: A Table Tennis Robot

Hintergrund: Tischtennis bietet ideale Bedingungen, um Kamera-basierte Roboterarme am Limit zu testen. Die besondere Herausforderung liegt in der hohen Geschwindigkeit des Spiels und in der großen Varianz von Spin und Tempo jedes einzelnen Schlages. Die bisherige Forschung mit Tischtennisrobotern beschränkt sich jedoch auf einfache Szenarien, d.h. auf langsame Bälle mit einer geringen Rotation. Forschungsziel: Es soll ein lernfähiger Tischtennisroboter entwickelt werden, der mit dem Spin menschlicher Gegner umgehen kann. Methoden: Das vorgestellte Robotersystem besteht aus sechs Komponenten: Ballpositionserkennung, Ballspinerkennung, Balltrajektorienvorhersage, Schlagparameterbestimmung, Robotertrajektorienplanung und Robotersteuerung. Zuerst wird der Ball mit traditioneller Bildverarbeitung in den Kamerabildern lokalisiert. Mit iterativer Triangulation wird dann seine 3D-Position berechnet. Aus der Kurve der Ballpositionen wird die aktuelle Position und Geschwindigkeit des Balles ermittelt. Für die Spinerkennung werden drei Methoden präsentiert: Die ersten beiden verfolgen die Bewegung des aufgedruckten Ball-Logos auf hochauflösenden Bildern durch Computer Vision bzw. Convolutional Neural Networks. Im dritten Ansatz wird die Flugbahn des Balls unter Berücksichtigung der Magnus-Kraft analysiert. Anhand der Position, der Geschwindigkeit und des Spins des Balls wird die zukünftige Flugbahn berechnet. Dafür wird die physikalische Diffenzialgleichung mit Gravitationskraft, Luftwiderstandskraft und Magnus-Kraft schrittweise gelöst. Mit dem berechneten Zustand des Balls am Schlagpunkt haben wir einen Reinforcement-Learning-Algorithmus trainiert, der bestimmt, mit welchen Schlagparametern der Ball zu treffen ist. Eine passende Robotertrajektorie wird von der Reflexxes-Bibliothek generiert. %Der Roboter wird dann mit einer Frequenz von 250 Hz angesteuert. Ergebnisse: In der quantitativen Auswertung erzielen die einzelnen Komponenten mindestens so gute Ergebnisse wie vergleichbare Tischtennisroboter. Im Hinblick auf das Forschungsziel konnte der Roboter - ein Konterspiel mit einem Menschen führen, mit bis zu 60 Rückschlägen, - unterschiedlichen Spin (Über- und Unterschnitt) retournieren - und mehrere Tischtennisübungen innerhalb von 200 Schlägen erlernen. Schlußfolgerung: Bedeutende algorithmische Neuerungen führen wir in der Spinerkennung und beim Reinforcement Learning von Schlagparametern ein. Dadurch meistert der Roboter anspruchsvollere Spin- und Übungsszenarien als in vergleichbaren Arbeiten.Background: Robotic table tennis systems offer an ideal platform for pushing camera-based robotic manipulation systems to the limit. The unique challenge arises from the fast-paced play and the wide variation in spin and speed between strokes. The range of scenarios under which existing table tennis robots are able to operate is, however, limited, requiring slow play with low rotational velocity of the ball (spin). Research Goal: We aim to develop a table tennis robot system with learning capabilities able to handle spin against a human opponent. Methods: The robot system presented in this thesis consists of six components: ball position detection, ball spin detection, ball trajectory prediction, stroke parameter suggestion, robot trajectory generation, and robot control. For ball detection, the camera images pass through a conventional image processing pipeline. The ball’s 3D positions are determined using iterative triangulation and these are then used to estimate the current ball state (position and velocity). We propose three methods for estimating the spin. The first two methods estimate spin by analyzing the movement of the logo printed on the ball on high-resolution images using either conventional computer vision or convolutional neural networks. The final approach involves analyzing the trajectory of the ball using Magnus force fitting. Once the ball’s position, velocity, and spin are known, the future trajectory is predicted by forward-solving a physical ball model involving gravitational, drag, and Magnus forces. With the predicted ball state at hitting time as state input, we train a reinforcement learning algorithm to suggest the racket state at hitting time (stroke parameter). We use the Reflexxes library to generate a robot trajectory to achieve the suggested racket state. Results: Quantitative evaluation showed that all system components achieve results as good as or better than comparable robots. Regarding the research goal of this thesis, the robot was able to - maintain stable counter-hitting rallies of up to 60 balls with a human player, - return balls with different spin types (topspin and backspin) in the same rally, - learn multiple table tennis drills in just 200 strokes or fewer. Conclusion: Our spin detection system and reinforcement learning-based stroke parameter suggestion introduce significant algorithmic novelties. In contrast to previous work, our robot succeeds in more difficult spin scenarios and drills

eWand: A calibration framework for wide baseline frame-based and event-based camera systems

Accurate calibration is crucial for using multiple cameras to triangulate the position of objects precisely. However, it is also a time-consuming process that needs to be repeated for every displacement of the cameras. The standard approach is to use a printed pattern with known geometry to estimate the intrinsic and extrinsic parameters of the cameras. The same idea can be applied to event-based cameras, though it requires extra work. By using frame reconstruction from events, a printed pattern can be detected. A blinking pattern can also be displayed on a screen. Then, the pattern can be directly detected from the events. Such calibration methods can provide accurate intrinsic calibration for both frame- and event-based cameras. However, using 2D patterns has several limitations for multi-camera extrinsic calibration, with cameras possessing highly different points of view and a wide baseline. The 2D pattern can only be detected from one direction and needs to be of significant size to compensate for its distance to the camera. This makes the extrinsic calibration time-consuming and cumbersome. To overcome these limitations, we propose eWand, a new method that uses blinking LEDs inside opaque spheres instead of a printed or displayed pattern. Our method provides a faster, easier-to-use extrinsic calibration approach that maintains high accuracy for both event- and frame-based cameras

Chemical patterns of colony membership and mother- offspring similarity in Antarctic fur seals are reproducible

Tebbe J, Humble E, Stoffel MA, et al. Chemical patterns of colony membership and mother-offspring similarity in Antarctic fur seals are reproducible. PeerJ. 2020;8: e10131.Replication studies are essential for evaluating the validity of previous research findings. However, it has proven challenging to reproduce the results of ecological and evolutionary studies, partly because of the complexity and lability of many of the phenomena being investigated, but also due to small sample sizes, low statistical power and publication bias. Additionally, replication is often considered too difficult in field settings where many factors are beyond the investigator’s control and where spatial and temporal dependencies may be strong. We investigated the feasibility of reproducing original research findings in the field of chemical ecology by performing an exact replication of a previous study of Antarctic fur seals (Arctocephalus gazella). In the original study, skin swabs from 41 mother-offspring pairs from two adjacent breeding colonies on Bird Island, South Georgia, were analyzed using gas chromatography-mass spectrometry. Seals from the two colonies differed significantly in their chemical fingerprints, suggesting that colony membership may be chemically encoded, and mothers were also chemically similar to their pups, hinting at the possible involvement of phenotype matching in mother-offspring recognition. In the current study, we generated and analyzed chemical data from a non-overlapping sample of 50 mother-offspring pairs from the same two colonies 5 years later. The original results were corroborated in both hypothesis testing and estimation contexts, with p-values remaining highly significant and effect sizes, standardized between studies by bootstrapping the chemical data over individuals, being of comparable magnitude. However, exact replication studies are only capable of showing whether a given effect can be replicated in a specific setting. We therefore investigated whether chemical signatures are colony-specific in general by expanding the geographic coverage of our study to include pups from a total of six colonies around Bird Island. We detected significant chemical differences in all but a handful of pairwise comparisons between colonies. This finding adds weight to our original conclusion that colony membership is chemically encoded, and suggests that chemical patterns of colony membership not only persist over time but can also be generalized over space. Our study systematically confirms and extends our previous findings, while also implying more broadly that spatial and temporal heterogeneity need not necessarily negate the reproduction and generalization of ecological research findings

WHO/IUIS Allergen Nomenclature: Providing a common language

A systematic nomenclature for allergens originated in the early 1980s, when few protein allergens had been described. A group of scientists led by Dr. David G. Marsh developed a nomenclature based on the Linnaean taxonomy, and further established the World Health Organization/International Union of Immunological Societies (WHO/IUIS) Allergen Nomenclature Sub-Committee in 1986. Its stated aim was to standardize the names given to the antigens (allergens) that caused IgE-mediated allergies in humans. The Sub-Committee first published a revised list of allergen names in 1986, which continued to grow with rare publications until 1994. Between 1994 and 2007 the database was a text table online, then converted to a more readily updated website. The allergen list became the Allergen Nomenclature database (www.allergen.org), which currently includes approximately 880 proteins from a wide variety of sources. The Sub-Committee includes experts on clinical and molecular allergology. They review submissions of allergen candidates, using evidence-based criteria developed by the Sub-Committee. The review process assesses the biochemical analysis and the proof of allergenicity submitted, and aims to assign allergen names prior to publication. The Sub-Committee maintains and revises the database, and addresses continuous challenges as new “omics” technologies provide increasing data about potential new allergens. Most journals publishing information on new allergens require an official allergen name, which involves submission of confidential data to the WHO/IUIS Allergen Nomenclature Sub-Committee, sufficient to demonstrate binding of IgE from allergic subjects to the purified protein

Cabbage and fermented vegetables : From death rate heterogeneity in countries to candidates for mitigation strategies of severe COVID-19

Large differences in COVID-19 death rates exist between countries and between regions of the same country. Some very low death rate countries such as Eastern Asia, Central Europe, or the Balkans have a common feature of eating large quantities of fermented foods. Although biases exist when examining ecological studies, fermented vegetables or cabbage have been associated with low death rates in European countries. SARS-CoV-2 binds to its receptor, the angiotensin-converting enzyme 2 (ACE2). As a result of SARS-CoV-2 binding, ACE2 downregulation enhances the angiotensin II receptor type 1 (AT(1)R) axis associated with oxidative stress. This leads to insulin resistance as well as lung and endothelial damage, two severe outcomes of COVID-19. The nuclear factor (erythroid-derived 2)-like 2 (Nrf2) is the most potent antioxidant in humans and can block in particular the AT(1)R axis. Cabbage contains precursors of sulforaphane, the most active natural activator of Nrf2. Fermented vegetables contain many lactobacilli, which are also potent Nrf2 activators. Three examples are: kimchi in Korea, westernized foods, and the slum paradox. It is proposed that fermented cabbage is a proof-of-concept of dietary manipulations that may enhance Nrf2-associated antioxidant effects, helpful in mitigating COVID-19 severity.Peer reviewe

Nrf2-interacting nutrients and COVID-19 : time for research to develop adaptation strategies

There are large between- and within-country variations in COVID-19 death rates. Some very low death rate settings such as Eastern Asia, Central Europe, the Balkans and Africa have a common feature of eating large quantities of fermented foods whose intake is associated with the activation of the Nrf2 (Nuclear factor (erythroid-derived 2)-like 2) anti-oxidant transcription factor. There are many Nrf2-interacting nutrients (berberine, curcumin, epigallocatechin gallate, genistein, quercetin, resveratrol, sulforaphane) that all act similarly to reduce insulin resistance, endothelial damage, lung injury and cytokine storm. They also act on the same mechanisms (mTOR: Mammalian target of rapamycin, PPAR gamma:Peroxisome proliferator-activated receptor, NF kappa B: Nuclear factor kappa B, ERK: Extracellular signal-regulated kinases and eIF2 alpha:Elongation initiation factor 2 alpha). They may as a result be important in mitigating the severity of COVID-19, acting through the endoplasmic reticulum stress or ACE-Angiotensin-II-AT(1)R axis (AT(1)R) pathway. Many Nrf2-interacting nutrients are also interacting with TRPA1 and/or TRPV1. Interestingly, geographical areas with very low COVID-19 mortality are those with the lowest prevalence of obesity (Sub-Saharan Africa and Asia). It is tempting to propose that Nrf2-interacting foods and nutrients can re-balance insulin resistance and have a significant effect on COVID-19 severity. It is therefore possible that the intake of these foods may restore an optimal natural balance for the Nrf2 pathway and may be of interest in the mitigation of COVID-19 severity

Prediction of second neurological attack in patients with clinically isolated syndrome using support vector machines

The aim of this study is to predict the conversion from clinically isolated syndrome to clinically definite multiple sclerosis using support vector machines. The two groups of converters and non-converters are classified using features that were calculated from baseline data of 73 patients. The data consists of standard magnetic resonance images, binary lesion masks, and clinical and demographic information. 15 features were calculated and all combinations of them were iteratively tested for their predictive capacity using polynomial kernels and radial basis functions with leave-one-out cross-validation. The accuracy of this prediction is up to 86.4% with a sensitivity and specificity in the same range indicating that this is a feasible approach for the prediction of a second clinical attack in patients with clinically isolated syndromes, and that the chosen features are appropriate. The two features gender and location of onset lesions have been used in all feature combinations leading to a high accuracy suggesting that they are highly predictive. However, it is necessary to add supporting features to maximise the accuracy. © 2013 IEEE